This invention belongs to the field of structured light 3D scanners. 3D scanners acquire a three dimensional shape of the object and are used in multiple applications, such as: Security applications, where a 3D scan of the face is performed for the sake of further face recognition [1]; Medical applications, such as 3D scanning of the dental area (U.S. Pat. No. 5,386,292); robot navigation (U.S. Pat. No. 4,611,292); Numerous other established and emerging applications.
Conventional image is a projection of a three dimensional scene onto a two dimensional image plane, and therefore the depth information is lost. Consider FIG. 1. A point C on the image sensor 16 of the camera 14 may correspond to either point A or point B, or any other point, lying on the ray 17.
A structured light 3D scanner consists of two main parts: a pattern projector and a camera. A pattern projector projects one or more patterns onto the object. The patterns are designed to allow determining the corresponding projector ray from the acquired images. Knowing that point C corresponds to projector ray 18, allows determining the distance to the point A by triangulation.
The projected patterns can be in the visible or infrared band. If the projected patterns are in the visible band, then an uncontrolled ambient illumination may interfere with the pattern, reducing the system accuracy, the projected pattern hinders the simultaneous acquisition of conventional image. In the security applications, like 3D scanning of the human faces, a pattern in the visible band disturbs the scanned individuals and gives away the system operation.
The projected patterns in the infrared band allow overcoming the above limitations. However, use of the infrared illumination requires use of a special infrared-sensitive camera, and therefore a simultaneous acquisition of the image in the visible band requires an additional camera. This increases the size, weight and cost of a 3D scanning system.
The projected patterns in the prior art are usually generated by the digital light projector [1,2]. FIG. 7 shows an infrared structured light 3D scanner in the prior art. A digital light projector 71 generates the patterns, an infrared filter 72 cuts off the visible band, and transmits only the infrared patterns; a signal reflected from the object 73 is divided by a semitransparent mirror 74 between the infrared camera 75, acquiring the patterns and the visible band camera 76, acquiring conventional images. This system is relatively expensive, bulky, and energy consuming, which prohibits many applications of the prior art 3D scanners.
The object of this invention is to provide a design of a structured light 3D scanner, comparable in the size and manufacturing cost with the simplest webcams, and possessing the webcam functionality. In other words the object of this invention is to add a 3D scanner functionality to a webcam, with only negligible increase of its size and manufacturing cost.
The present invention is a structured light 3D scanner, consisting of a camera with a specially designed image sensor and a specially designed pattern projector. An image sensor is designed so, that some of its pixels are sensitive in the visual band, while some other pixels are sensitive in the infrared band. This allows acquiring a conventional image and a stealth 3D scan in the infrared band by a single camera in the system.
In prior art color image sensors, the green pixels are redundant, since for each one blue and one red pixel there are two green pixels, as shown on FIG. 2. Therefore, converting one of the green pixels into the infrared pixel does not decrease the sensor resolution. The infrared sensitive pixels are created via the deposition of infrared transmitting filter.
When the image is acquired by an invented image sensor, the information from the visible band pixels is used to construct a conventional image, while the information from the infrared pixels is used to construct an infrared image. We call an invented image sensor a “dual band” image sensor, since it simultaneously acquires images in the infrared and in the visible bands. The dual band image sensor requires only one additional type of filter, and its manufacturing cost is only negligibly higher than the manufacturing cost of conventional sensor. Use of dual-band image sensor does not requires a change of the camera design, since it has the same interface with the camera as the conventional image sensor; the camera acquires the full frame picture, containing both the infrared and the visible band pixels, which are discriminated in the image processing.
An invented fixed pattern projector has a single pattern mask and several LEDs behind it. Switching between the LEDs shifts the projected pattern, as shown on FIG. 6. One of the particularly efficient patterns for structured light 3D scanners is a set of three phase-shifted sine patterns ([2], U.S. Pat. No. 6,559,954, U.S. Pat. No. 6,438,272). In specific embodiment of this invention, three phase shifted sine patterns are generated by a pattern mask with sine profile on it and three infrared LEDs behind the mask. The invented projector is compact, measuring only several millimeters in each dimension and has low manufacturing cost.
The 3D scanner using the disclosed designs of the dual-band image sensor and the fixed pattern projector has functionality of a conventional camera and an infrared 3D scanner, while its size and manufacturing cost are almost same as of the camera alone.